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The Use of Systems Analysis and Management Techniques 
In Program Planning and Evaluation^ 



Desmond L. Cook 



Introduction 



It gives me a great deal of pleasure to have been Invited to make 
a presentation at this symposium for two reasons. First, I welcome very 
much the opportunity to become personally acquainted with each of you and 
to acquire greater familiarity with the overall dimensions of the 
Operation PEP project. Second, I am In agreement with the general idea of 
Operation PEP that systems approaches offer a significant means for bringing 
about improvements in the educational environment. 

Dr. Estes has highlighted the need for planned change in education 
in his keynote address. As I have listened to this concept being bantered 
about during the last couple years, I fear, that It Is very similar to 
the comment about the weather attributed to Mark Twain. That is, every- 
body is talking about planned change but very few seem to be doing anything 
about it. My knowledge of the work that is going on under the duration 
of Drs. Corrigan, Kaufman, and Miller represents, In my thinking, one of 
the few systematic and concentrated efforts to develop the needed leader- 
ship and to provide the techniques to bring about desired changes on a 
planned basis in the area of education. 

The development of the concepts and procedures underlying Operation 
PEP turns out to be a good case of foresight. This remark Is made in 
view of the fact that within the last couple of months I have become 
involved with a U. S. Office of Education project concerned with the 
establishment of discreet but comprehensive planning agencies within a 
group of seven states. It Is hoped that the process of developing and 
establishing these state planning agencies could then serve as a model 
for the other states to follow. In thinking about this project. It seems 
to me that the Ideas behind Operation PEP are both highly relevant to 
and timely for this projected activity. The existence of both Operation 
PEP and the seven state project serve only to reinforce my thinking 
that if any significant changes are to be made In education within the 
school districts, each state, and throughout the country, systematic 
planning efforts offer the greatest opportunity to accomplish this goal. 
Unfortunately, as desirable as our goal is, the educational leadership 
is handicapped by a lack of comprehensive skills and techniques through 
which it can achieve the goal. It Is my understanding that the purpose 
of the project In which many of you have been participating during the 
past months is to remedy this situation by providing the educational 
leadership within each school district with techniques and skills which 
they can use to facilitate movement toward the goal. 

Drs, Kaufman and Corrigan have presented some of these techniques 
with their descriptions of systems analysis and synthesis. The task 
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assigned to me is to discuss the use of systems analysis and management 
techniques in program planning and evaluation. A more 
for my comments can be secured by noting that the presentation re ates 
primarily to the step labeled "Establishing Management and Contro Sub- 
System" in their total list of steps. To avoid some 

to deal primarily with the purpose and functions of management techniques 
as they relate to program planning and evaluation. 

The presence of such a topic in this symposium is important because 

management problems are an inherent aspect of any j ® ? i\ 

This statement is reinforced by the recent observations of Andr^ (I) on 

the management probieras In applied social f®®®f 

that the payoff from much social research has been limited because of 
various problems which can be classified into two major types of stres.es-- 
those inherent In managerial arrangement and those growing out of prosram 
and research demands which he labels the arofessional conte^. Time does 
not permit a detailed presentation of his comments relative to each type. 
For the moment, it is perhaps sufficient to say that his observations are 
not unique since there appears to be little done In the P''®P®';®^’°" 
researchers, let alone school administrators, with regard to the processes 
of management although both groups are often highly prepared In Their pr - 
fessional substantive area. Consequently, "»"y/®Tion programs fa i to 
reach full fruition. To prevent needless repetition of this situation 
is appropriate that some time be given to the topic of management tech- 
niques when talking about programmatic efforts. 

The concept of management techniques has several meanings. Let 
■„>e state that this presentation will not focus upon techniques «;®]®Ting 
to topics such as how to deal with personnel problems, office organization, 
or methods of financial accounting. Instead, the focus J’®‘1P°'' J_® 
general concept of management systems and within this concept, the nature 
and function of management information systems. During the course o 
presentation, an attempt will be made to Interrelate the concepts of 
systems analysis and program planning and evaluation. 



Program Planning and Evaluation 



To provide a context for the subsequent discussion of management 
techniques, 1 should like to review briefly the concept of program 
pl3nhtnv3 and evaluation. 



It Is very much "In" today to talk about programsatlc thrusts in 
various fields. We lear about the space program, <‘®^'®"*® PT^ST®";; J'®®®®!'®^ 
programs, air pollution programs, and so on. Like many ®°"®®P*® *"®* 

are "In" no one seems fully to have reached consensus as to e^ctly what 
is being talked about. My remarks will probably not solve this problem 
but I would like to give you my idea of what Is being talked about when 
the topic is program planning and evaluation. 



Whenever program Is mentioned, I think primarily of ® ®®T®J^y 
organized effort utilizing a "critical mass" of resources moving toward 
the achievement of an objective which the culture has agreed upon as 
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being worthy of being attained. The elimination of polluted air, poverty, 
dreaded diseanes like leukemia, are worthy program objectives. It can 
almost be axiomatic that when there is no objective there Is no meaningful 
program. Having established the goal, a plan is developed to accomplish 
it. Briefly, a blueprint or a roadmap for the future is constructed. In 
doing so, an attempt Is made to build the plan to conform to the parameters 
of time cjnd cost that it will take to achieve the performance level 
stated in the objective. Having established the plan, the program moves 
into an operational phase which Is essentially the carrying out of the 
activities that are expected to lead to the objective. As work moves 
along and tasks are accomplished, there is a need to evaluate progress to 
see how well the actual work is going in terms of how it was planned to 
go. Such questions as Is the program ahead or behind schedule? Is the 
rate of expenditures too fast? Vihere are problems developing? are 
illustrative questions which are raised In evaluating program status. I 
would call your attention at this time to the fact that evaluation Is 
being defined here in terms of what management refers to as process 
control as opposed to concerns about the quality of the end product, it 
is through the comparison of what should be happening to what Is actua 1 ly 
happening that problem areas are identified for subsequent management 
actions during the course of the program. V/lth this brief background 
on program planning and evaluation, let us turn our attention to the topic 
of management techniques. 



Management Techniques 

The success of any programmatic effort will depend greatly not 
only upon the initial planning effort that goes into It, but just as 
importantly, upon the employment of a viable management plan by the 
persons charged with the responsibility for carrying out the program 
effort. The success of such a large scale program as the lunar landing 
Is due not only to the professional technical and scientific competence 
absembled for the program but also to the managerial competencies and 
skills brought to the program. This latter point Is often overlooked by 
most of the general public until there is a disaster such as the recent 
Apollo fire when the program management was subjected to heavy and, in my 
opinion, much undeserved criticism. 

To talk about management techniques without some general reference 
to the nature and function of management would be an incomplete activity 
since such techniques or tools are basically designed to assist management 
in carrying out Its responsibilities. There are many definitions of 
management but basically most definitions focus upon the responsibilities 
of organizing, directing, and controlling personnel and resources to 
carry out the accomplishment of an objective. 

It is also generally recognized that a manager’s principal functions 
are those of problem Identification, decision-making, and the prevention 
of future problems. Our present state of knowledge about these three 
functions Is that they cannot be accomplished without some systematic 
procedures. It Is perhaps therefore more appropriate to talk about 
management systems rather than management techniques. 
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It might be helpful to us If we wen> to define what Is meant by 
a management system. In their recent book on network based management 
systems, Archibald and Vlllorla (2) provide us with a useful working 

def I n i 1 1 on . 



V/e may define a management system as a set of operating 
procedures which personnel carry out to acquire needed 
information from appropriate sources, process the data 
In accordance with a pre-programmed rationale, and 
present them to decision makers In a timely, meaningful 
form. Most contemporary systems Involve manual "data 
collection and Input, machine processing, tabular and 
graphic output production, and human analysis and 
interpretation# Thus we can say that the systems collect, 
synthesize, process, transmit, and display Information, 
which flows from a primary source, through an editing, 
computation, and selection process to the manager. 



Two principal ideas are highlighted In this definition, 
systems are designed to provide information. Second, the final dec is on- 
making operation Is left to the manager. Some sophisticated systems do 
have what Is called pre-programmed declslon-maklng as an Inherent part or 
them. Most systems, however, still rely upon the use of humans to make 
non-prog rammed decisions. Further, most of the systems °P® 

primarily aimed at facilitating the process control responsibility given 
to a manager. The focus of control Is nevertheless related to the 
planning effort that has gone on before. It can be almost stated as an 
axiom that If one establishes a plan, he Intends to exercise control to 
make sure that the plan Is accomplished. 



In view of the definition presented above. It seems more appropriate 
to label such systems as management Infor.natlon systems. To visualize 
how such systems operate within the management situation, picture a tr ang e 
divided Into approximately three equal horizontal sections. The 
section of this triangle consists of elements of Informatlon--or what Is 
commonly referred to as the data base. What and how much information should 
be In this base Is open to question but generally within a program planni g 
effort the Information assembled relates to time, cost, and performance. 

Such Information Is Included because most program efforts operate w thin 
a set of tlme/cost/performance parameters. That Is, there is usual ly some 
goal to be achieved which has specified criteria of accompl Ishment and 
this goal must be reached within a schedule period of time and certain 
budget limitations. Most military weapons systems operate under such 
conditions as do many non-mil Itary programs such as the lunar landing or 
space programs. Many programs funded under the Elementary and Secondary 
Education Act also operate under a set of similar parameters. Repoi ts 
to management stem from this data base on schedule reporting periods 
(Ke., dally, weekly, monthly, etc.). In terms of management levels, 
this base section Is often equated with immediate performing departments 
or units concerned with the specific day-to-day tasks, or operation al 
control as It Is called ( 5 )* 
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The middle section of the triangle represents operating management 
whose primary concern is to assure that resources are obtained and used 
effectively and efficiently in accomplishing the objectives. To do this, 
information requests and demand reports are secured from the data ease. 

Such reports are secured often in addition to the schedule reports 
sent upward from the data base. The main function of this level Is that 

of control. 

The top section of the triangle represents general or executive 
management which is primarily concerned with policy level decision- 
making. Focus here Is often on information for the planning effort as 
contrasted to information for the middle section which Is focused upon 
control • There are times hov;ever when top-level management desires and 
uses the information for control purposes. 

V/lthIn this structure, different kinds of information (i.e., 
data) are needed by the three levels for different purposes. It Is 
important to know the information requirements at each management level 
along with the description of the data and how it Interrelates In order 
to establish a useful data base. 

It has been generally agreed upon that managers operating within 
the program or project planning and evaluation situation require 
or information relating to time or schedules, costs, or resources, and 
performance, reliability, or quality of objective accomplishment. Of 
these three types of data, the most common data obtained and used in a 
program situation is that of time or schedule. 

While seemingly easy to think about, the development and use of 
management systems is not so easy because a system designed for one 
purpose may not be suitable for a different purpose. For this reasoiii 
numerous management systems have been developed over the past several 
decades to facilitate the manager's task. Three general types of systems 
have been developed each designed to serve a different purpose. One 
group of systems relates to the quality characteristics of a product, 

A second category of systems related to the operations involved in 
producing the product, A third group of systems relate to the admin- 
istration Involved in carrying out the operations. 

It would not be possible to discuss all three types of systems 
within the time allotted so I have chosen to talk primarily about 
selected operations -re la ted systems which have become increasingly popular 
during the past decade because of their relatively. high degree of success 
in carrying out research and development activities within the mnitary- 
industrial complex of our society. The application of such techniques 
to the field of educational research and development is just beginning 
but their value has already been demonstrated and they should grow in 
increased usage during the next decade. 
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Network-based Ma na qement $ vs terns 



Any formal or structured management technique that Is to be mean- 
ingful and useful for programs of planned change must help us to accomplish 
three tasks (3). The first task Is to develop a general program model 
based upon a logic derived from the substance of the work to be performed. 
Major program elements are then identified which in turn are further 
reduced into smaller work packages. The second task Is to sequence the 
order of effort by determining the logical relationships between the 
events. It should be recognized that within the research and development 
situation, any such ordering will be tentative and be subjected to 
continuous modification. The third task is to provide a time frame so 
as to establish a basis for determining present and projected resource 
requirements and to provide a schedule for the conipletlon of events and 
activities. Further, any such technique must be easily understood, help 
to establish priorities, deal with the uncertainty problem, be predictive 
and help to forecast problem areas in advance, and enable us to manage 

by exception. 



The specific group of process -re la ted management systems that 
will helo us to accomplish the above tasks are those referred to as 
network-based management systems. The two most popularly kn^n 
of these systems are those of the Program Evaluation and Review Technique 
(or PERT) and the Critical Path Method (or CPM) . While each or these 
systems has distinctive characteristics, they do have sufficient 
similarity that both can be discussed under the more general concept of 
network techniques. Time does not permit a detailed presentation of 
system characteristics so only an overview can be presented here. 



The implementation of network techniques as a management information 
system for a research program or project can be subdivided into two 
principal steps for discussion purposes. The first Is planni . o a and the 
second is control . Operationally, both are highly Interdependent and 
go on concurrently. 



The first step in planning Is to breakdown or breakout the work^ 
that has to be accomplished in order to achieve the prime and supporting 
objectives. This process is referred to as establishing the wo r ^ 
breakdown structure. The process Is a top-down activity with prime 
objectives at the top being broken down Into successive smaller and 
smaller units until some point is reached at which there seems to be 
no additional value in breaking out the particular tasks to be done. 

The final unit of breakdown is referred to as a work package, 
at this point that the work breakdown structure rocess and the mission 
profile derived from a systems analysis procedu become highly Interrelated, 
Once the analysis program has been carried thro^jh the mission, function, 
and task analyses stages, the work breakdown structure required for 
network planning should have been pretty well established. A major 
Integration of systems analysis and manag^ent techniques for program 
planning and evaluation takes place at this point. 
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Having once identified the various work packages that have to 
be done to accomplish the mission, a network Is established. The network 
is a graphical representation of the plan showing the logical sequence 
and interdependency of work to be accompiished from the time the program 
is initiated until Its final termination. Individual tasks to be 
accomplished which utilize resources and consume time are called actj^- 
i ties . Activities are usually represented on the network by a straight 
line with an arrowhead to show flow. The start and/or completion of an 
activity Is referred to as an event . Events are points on time and 
do not consume time or resources. They are represented on the network 
by a circle or other geometric figure. The work flow In the network Is 
always from left to right. The amount of detail to be Included In a 
network is a function of Its purpose. Operating networks wlii have more 
detail than networks prepared for top management use. 

The network serves many functions but among the principal ones are 
a graphical representation of tlis program plan, a communication tool for 
the performing and managerial staff Involved, and a basis for control by 
management. 

Once the network is finalized, the next step is to establish a 
time frame for the total project or program duration. Depending on 
which specific network system (r>tRT or CPM) Is to be employed, single 
or multiple time estimates (ucually three) are secured for each of the 
Individual tasks assuming resources as planned or available and under 
a normal resource application rate (e.g, 40 hr. week) from those persons 
who are to do the task. Single estimates are referred to as deterministic 
estimates while multiple estimates are known as probabilistic estimates. 
The latter estimating procedure has the advantage of helping us to deal 
with the uncertainty problem which characterizes much research and 
development work. When three time estimates are secured, an average 
estimate is obtained along with a measure of the variability of the 
estimates by the application of appropriate formulas. 

After the single or average time estimates are secured for each 
task, they are utilized to find the total time for the project as well as 
the time needed for the start/complet ion of each event In the network. 
There Is one set of tasks In the network that Is the most time consuming. 
This set of tasks Is referred to as the critical path, in addition to 
the critical path, we are able to determine the amount and location of 
slack existing within the netv/ork. Slack refers to the difference between 
the earliest time an event can take place and the latest allowable time 
it can take place without jeopardizing project completion. While diamonds 
may be a girl's best friend, slack Is one of the best friends a manager 
can have. 

Once the time frame Is established, schedule dates are set up for 
the start and completion of work after consideration Is given to resource 
requirements needed for the various activities In the total program or 
project and their availability. Once the schedule has been established, 
work on the project now begins. 
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Successful completion of the above tasks done in the planning 
stage provides the information needed for the data base of the triangle 
referred to earlier. As I prefer to say, the "shoujds" for the project 
or program have been established. 

The second stage of implementation, that of control, begins once 
work on the project is initiated. Periodic reports are prepared which 
reflect actua 1 status of the project schedules with regard to work 
completed, work in progress, and work yet to be done. These reports are 
summarized and presented to management in both graphic and narrative 
form with a primary emphasis upon the exception-reporting principle. 

That is, the reports presented to management are so organized that only 
the most serious problems (i.e., deviation of performance from plan or 
actuals from shoulds) are presented for management consideration and 
decision. Solutions to these problems usually take the form of adding 
resources, redefining tasks, eliminating tasks, or paralleling tasks 
which original ly were in linear order. Considerations of time/cost/ 
performance trade-offs are made at this time. Once management action is 
taken, necessary revision or changes in the project or program network 
are made, new time estimates secured, and new schedules established. 

V^ork then continues until the next reporting period and so on until the 
program or project objective is reached. 

The above is admittedly a brief description of network systems 
but additional information can be obtained from a reference such as that 
by Archibald and Villoria (2) or from a monograph ( 4 ) describing the 
applicability of network systems to the field of education and distributed 
by the U.S. Office of Education. I hope that sufficient description 
has been provided so that you can see that network techniques meet the 
requirements of the three tasks as set forth earlier and meet criteria 
for a management information system. It should be pointed out here that 
network systems have been expanded to include the planning and control of 
costs with some exploratory work being done to include the planning 
and control of performance but time again prevents us from presenting 
details on these procedures. 

it can be stated that network techniques with their analytical 
and diagrammatic approaches to the problem of planning and control 
assist management with the following kinds of tasks: 

1. defining the work to be carried out 

2. producing better schedules based on available and 
needed resources 

3. making decisions about the best way to apply resources 
to achieve program objectives 

4. monitor progress and identify those points where delays 
could jeopardize the project in time to permit corrective 
action to be taken (6). 
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It Is only fair to point out that not everyone is in agreement 
with the idea that management systems in general and network techniques 
in particular have any value for planning educational changes. Such 
objections center on the idea that one cannot identify the objectives 
as precisely as the system requires, that many of the tasks to be 
accomplished are not determinable let alone definable so that useful 
time estimates can be secured, and that one cannot plan n:*r schedule 
intellectual and creative work. Such objections may have validity for 
many research and development programs, even including those in the field 
of education. The successful employment of such techniques In education 
has, however, demonstrated that network systems do have validity and 
relevance for many education problems. Coupled with system analyses 
techniques, network systems are highly potent tools for bringing about 
desired improvements in education. 



An 1 1 lustration 



The concepts of program planning and evaluation, managerial 
techniques, and their relations to systems analysis procedures have been 
reviewed briefly. You might well ask how do these all operate con- 
jointly to bring about changes in the field of education. 

Let me try to answer this question by using an over-simplified 
illustration. Let us suppose that you are .concerned with the im- 
provement (or change to use present day pedagese) of student achievement. 
Through systems analysis, you have established criteria of performance 
which are quantified for later evaluation. Having taken this step, 
you now consider the means-ends alternatives available to you. One 
could employ televised Instruction, computer-assisted instruction, 
team teaching, and so ori. Let us assume that the results of your 
deliberation indicate that team teaching is the highest ranking al- 
ternative (notice that I do not say best). So your mission objective^ 
is to implement team teaching in your school district. Through function 
and task analysis, the many functions and tasks that have to be carried 
out during the period of implementation have been identified and the 
mission profile prepared. 

At this stage of the game, you are now ready for a management 
system to help you with the Job of actual planning and controlling 
the implementation. Using network systems, the work breakdown structure 
is established, the network drawn, time estimates secured, schedules 
set up, and resources allocated, critical milestones identified, and 
reporting systems prepared. Once under way, the manager will have to 
evaluate and review how the implementation process is going. Is he 
ahead or behind schedule? Because of unforeseen problems such as a 
strike by the local teachers or the nonavailability of a consultant, 
are changes on the plan required? Periodic reports prepared for 
management will not only appraise him of present problems but will also 
identify potential future problems. Carefully prepared progress reports 
and their proper utilization by our school district superintendent will 
go a long way in making sure that the implementation of the team teaching 
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program will have actually taken place by the date he has established 
and within the projected costs. 

V/hile over*s impl if ied| the above illustration does incorporate the 
three concepts that I have been asked to deal with this afternoon. 

Systems analysis Is used to help decide what the objective is, the most 
feasible alternative to achieve It, and the many tasks which must be 
done along with their logical sequence. Management techniques will help 
us to plan the program In terms of how we intend to proceed and then 
help us to control or stay on top of the operation. Using this technique, 
we can bring together both professional substantive competence and 
managerial skills so that our programs of planned change can become 
living evidence of our efforts. 

Conclusion 

To conclude this presentation, it seems Imperative to me that If 
we are going to bring about Improvements or changes In the field of 
education they must, of necessity, be both planned and controlled. T e 
employment of systems analysis and synthesis procedures offer a challenge 
and an opportunity to Improve our planning effort. Such techniques force 
us to face up to the question of what exactly Is it that we want to 
accomplish and how we Intend to go about it. The specification of the 
objective and Its subsequent analyses to Identify the functions and tasks 
which have to be accomplished In order to reach the objective require us 
to use our logical skills to a very high degree. 

Once the analysis Is made, we are only partly along the road. The 
operations necessary to bring about the change or accomplish the objective 
have to be Initiated and carried out. This situation requires selection 
and employment of some management system or technique to make sure that 
the plan Is accomplished within the tlme/cost/performance constra .nts 
that are present or established. Employment of new and highly successful 
managemcsit techniques commonly called network-based systems, specifically 
PERT and Critical Path Method, provide a means of meeting this end. 

The focus of this symposium Is on planning for educational change 
and on the preparation of educational planners. As noted above, planning 
Implies control. Because of this relationship, educational planners 
should be knowledgeable about and competent In the tools and techniques 
of management if there is to be assurance that the program goals that 
have been established are to be accomplished. I hope that my r^rks 
today have stimulated each of you to become Interested In securing 
proficiency with such management techniques. 
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